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gacmMB  PAlSl  BUMt-WOr  FI 

I.  INTRODUCTION 
A.  Lynn  Misselt 

An  assessment  of  student  achievement  and  of  student  and 
instructor  attitudes  during  the  period  from  January,  1975, 
to  September,  1975,  was  made  by  the  U.S.  Air  Force  Human 
Resources  Laboratory  (AFHRL)  and  the  Air  Training  Command 
(ATC)  as  part  of  their  jointly-conducted  Service  Test  of  the 
PLATO  IV  System  at  Chanute  Air  Force  Base.  Analyses  of  the 
achievement  and  attitude  data  were  carried  out  by  the 
Military  Training  Centers  (MTC)  and  PLATO  Educational 
Evaluation  and  Research  (PEER)  groups  of  CERL  and  will  be 
reported  as  products  of  the  AFHRL/ATC  evaluation  effort.  It 
is  the  understanding  of  the  MTC  and  PEER  groups  that  the 
AFHRL/ATC  reports  will  treat  the  instructional  effectiveness 
and  instructional  impact  results  of  the  Service  Test  inde¬ 
pendently  and  hence  will  not  attempt  to  relate  achievement 
results  to  attitudes.  We  believe,  however,  that  a  complete 
examination  of  the  achievement-attitude  relationship  will 
enhance  the  understanding  of  the  outcomes  of  the  Chanute 
Service  Test  and  may  lead  to  useful  generalizations  or 
hypotheses.  We  have  planned,  therefore,  a  series  of 
additional  analyses  to  relate  attitudinal  and  achievement 
outcomes . 

Database  Available  from  Chanute  Service  Test 


In  order  to  assess  the  effectiveness  of  the  PLATO  IV 
system  in  a  military  training  environment,  personnel  at 
Chanute  AFB  developed  a  set  of  PLATO  lessons  covering 
cognitive  elements  of  a  course  segment  in  elementary  auto¬ 
motive  principles.  This  six-week  segment  formed  a  basic 
common  core  of  material  for  four  special  purpose  vehicle 
repairman  courses.  That  is,  students  assigned  to  each  of 
the  four  specialty  courses  all  received  similar  instruction 
in  the  fundamentals  of  automotive  mechanics  before  branching 
off  for  additional  instruction  in  the  maintenance  of  vehi¬ 
cles  in  their  own  specialty  (e.g.,  fire  trucks,  tow  trucks, 
endloaders,  etc.).  This  common  course  segment  was  taught  in 
four  units  or  blocks,  averaging  1  1/2  weeks  in  length,  with 
a  Block  Exam  administered  at  the  end  of  each  block. 

PLATO  was  introduced  in  the  four  target  courses  on  a 
staggered  time  schedule  so  that  classes  entering  one  of  the 
courses  began  to  use  PLATO  as  an  integral  part  of  their 
instruction  as  early  as  January,  1975,  while  conventional 
instruction  was  retained  in  the  other  courses  until  later  in 
that  year.  By  October,  1975,  all  special  purpose  vehicle 
repairman  students  were  receiving  instruction  via  what 
Chanute  AFB  calls  the  PLATO-Eased  Training  System.  Under 
this  system,  PLATO  lessons  comprised  1/3  to  1/2  of  the 
instruction  in  the  six-week  "common  course  segment."  The 
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remaining  instruction  was  presented  by  programmed  texts, 
films,  and  laboratory  exercises. 

The  CBE  materials  used  for  mainline  instruction  con¬ 
sisted  of  thirty  five  on-line  lessons,  totaling  about  20 
hours  in  length.  Each  of  these  lessons  had  a  criterion- 
referenced  test  at  the  end  which  was  called  a  master 
validation  exam  (MVE).  Students  who  failed  to  achieve  a 
passing  score  on  a  lesson’s  MVE  were  required  to  review  the 
lesson  and  to  repeat  the  MVE  until  they  scored  above  the 
minimum  criterion  for  mastery. 

Treatment  conditions.  During  the  period  from  January, 
1975,  until  September,  1975,  members  of  the  joint  AFHRL/ATC 
evaluation  team  collected  achievement  and  attitude  data  on 
students  under  three  treatment  conditions.  The  first  of 
these,  the  PLATO-Based  or  PB  condition,  consisted  of  instuc- 
tion  with  the  PLATO-Based  Training  System  on  the  cognitive 
content  of  the  common  course  segment.  In  the  second  condi¬ 
tion,  the  Conventional-PLATO  or  CP  condition,  instructors 
assigned  many  of  the  PLATO  lessons  to  their  students  but 
also  presented  the  same  material  via  conventional  lecture 
methods  in  the  classroom.  Students  in  the  third  condition 
received  no  PLATO  instruction  and  hence  were  designated  as 
the  Non-PLATO  or  NP  group. 

Achievement  data.  The  basic  achievement  data  collected 
for  the  AFHRL/ATC  evaluation  from  January,  1975,  to  Septem¬ 
ber,  1975,  included  scores  on  the  four  block  exams,  final 
grades  (which  were  actually  just  the  average  of  the  block 
exam  scores) ,  and  results  from  three  administrations  of  a 
special  topical  test  developed  by  the  AFHRL/ATC  evaluation 
team.  The  special  topical  test  contained  30  items  over  five 
topics  covered  by  PLATO  lessons  and  20  items  over  an 
additional  four  topics  that  were  not  presented  by  PLATO. 

The  test  was  administered  to  students  in  all  three  treatment 
conditions  on  three  different  occasions:  1)  as  a  Pretest  on 
the  first  day  of  instruction,  2)  as  a  posttest  immediately 
following  the  common  course  segment  (this  administration  was 
called  Posttest  _I ) ,  and  3)  at  the  end  of  each  of  the  four 
specialty  courses.  This  last  administration  was  called 
Posttest  II. 


Attidudinal  data.  The  student  attitude  questionnaire 
used  in  the  AFHRL/ATC  evaluation  is  reproduced  in  the 
Appendix  to  this  report.  The  items  in  Section  I  (called 
the  "short  form"  questionnaire)  were  administered  to  the 
PB  and  CP  students  after  each  block  of  instruction  in  the 
the  common  course  segment.  The  NP  students,  meanwhile,  were 
asked  to  complete  a  similar  set  of  items  which  dealt  with 
attitudes  toward  traditional  instruction  rather  than  toward 
computer-based  education.  The  items  in  Section  II  sought  to 
measure  attitudes  toward  the  PLATO-Based  Training  System 
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while  those  in  Section  III  asked  about  the  affective  status 
aroused  by  working  with  the  PLATO  system.  The  items  in 
Sections  II  and  III  were  collectively  referred  to  as  the 
"long  form"  questionnaire.  Because  it  assumed  experience 
with  computer-based  education,  the  long  form  questionnaire 
was  administered  only  to  the  two  groups  who  received  PLATO 
instruction.  It  was  administered  at  the  end  of  the  common 
course  segment  just  before  the  students  moved  on  into 
instruction  in  their  own  vehicle  repairman  specialties. 

The  attitudes  of  instructors  toward  the  PLATO  Based 
Training  System  were  also  measured  by  a  questionnaire. 
Unfortunately,  it  is  impossible  to  relate  instructor  atti¬ 
tudes  directly  to  the  attitude  or  performance  of  their  stu¬ 
dents  because  of  the  manner  in  which  instructors  were  giver, 
teaching  assignments  in  the  target  courses. 


II.  LONG  FORM  OF  ATTITUDINAL  DATA 
Kikumi  Tatsuoka 

In  order  to  simplify  the  66  attitudinal  items  into 
several  groups  of  related  items,  Factor  analysis  was  per¬ 
formed  on  the  scores  of  questionnare  items.  The  responses 
to  the  66  items  of  the  long  form  attitude  questionnaire  (45 
items  from  section  II,  20  items  from  section  III,  plus  one 
item  indicating  treatment  condition)  were  first  subjected  to 
a  Principal  Components  analysis.  The  largest  seven  eigen¬ 
values  were  20.4513,  4.0512,  2.2535,  1.8710,  and  1.6367. 

Then  the  factor  matrix  associated  with  the  largest  seven 
principal  components  was  rotated  by  the  Varimax  procedure. 

The  25  items  having  factor  loadings  larger  than  .5  in  abso¬ 
lute  value  were  extracted  as  a  common  factor  of  this  66-item 
analysis.  The  common  factor  had  a  variance  of  15.2554 
and  a  percentage  variance  of  27.1905,  while  the  total  sum  of 
commnualities  was  56.1056.  Indeed,  many  items  within  the 
questionnaire  were  clustered  together  as  a  common  factor. 
Thirty-nine  items  in  this  first  factor  had  loadings  larger 
than  .30  in  absolute  value. 

All  of  these  items  indicated  positive  attitudes  toward 
the  PLATO  program.  However,  a  close  examination  of  items  m 
this  factor  indicated  that  there  were  three  kinds  of  favor¬ 
able  attitudes  toward  PLATO:  the  first  was  personal  involve¬ 
ment  with  PLATO;  the  the  second  type  represented  the 
respondent's  perception  of  the  effectiveness  of  PLATO 
lessons;  the  third  was  more  affective,  with  responses  such 
as,  "Working  with  PLATO  is  fun,  enjoyable,  and  encouraging." 
The  items  from  Section  II  having  loadings  larger  than  ,70  in 
absolute  value  were  3,  5,  6,  13,  17,  18,  19,  35,  43,  and  44. 
These  items  probably  indicated  a  favorable  attitude  toward 
PLATO  without  any  noticeable  reservation.  Thus  they  will  be 
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designated  as  type  I  items.  The  items  having  lower  loadings 
also  from  Section  II  were  connected  with  the  second  kind  of 
attitude,  a  more  objective  statement  of  the  perception  of 
the  effectiveness  of  PLATO  lessons.  These  items  were  12,  22, 
33,  41 ,  and  45.  The  third  kind  of  attitudes  were  from 
Section  III  and  they  correlated  highly  with  both  the  first 
and  the  second  kinds  of  attitudes.  It  would  be  interesting 
to  know  to  what  extent  the  mechanistic  and  impersonal  CAI 
lessons  will  work  effectively  and  achieve  the  aimed  goal  of 
training . 

Analysis  of  the  items  in  Section  III.  Since  our 
interest  was  to  relate  the  attitude  responses  to  the  per¬ 
formance  results,  the  common  factor  from  this  analysis  seemed 
too  general  and  ambiguous  for  our  purpose.  Therefore  the 
45  items  in  section  II  and  question  2  in  section  III  were 
separated  from  the  items  in  Section  III.  Items  7-18  in  sec¬ 
tion  III  and  these  46  items  were  analyzed  independently  in 
the  same  way  as  the  66  items  were  analyzed.  The  result  of 
analysis  for  the  affective  items  7  through  18  in  Section  III 
are  shown  in  Table  1 .  The  eigen-values  of  the  first  three 
components  were  6.5203,  1.5518,  and  .7927,  and  the  cumulative 
percentage  of  the  first  two  components  was  68.6269. 


Table  1 

Variance  in  the  Two  Factor  Space  Accounted 

for  bv  each  Factor  Following  Varimax  Rotation 

Factor  %  of  variance  Cum.  of  %  of  Var. 

1  3-5720  37.4316 

2  2.9769  68.6269 


These  factors  were  renamed  FVI  and  FVII,  respectively,  and 
were  retained  in  further  analyses. 

The  analysis  of  the  items  in  Section  II.  The  45  items 
and  item  3  in  Section  III  were  factorized.  The  principal 
component  analysis  of  the  46  items  showed  eigen-values  of 
14.1561,  3-3001,  1.9100,  1.5955,  1.4763,  1-3580,  and  1.3020 
as  the  first  seven  components.  The  variance  of  the  first 
principal  component  was  still  large  relative  to  the  other 
components  and  the  percentage  of  variance  was  almost  31%. 
Varimax  rotation  was  then  performed  on  the  factor  matrix 
associated  with  the  first  seven  components,  which  accounted 
for  54.56%  of  the  variance  in  the  original  46  variables. 

The  results  are  shown  in  Tables  2  and  3.  The  common  factor 
from  this  46-item  analysis  was  very  similar  to  the  one  from 
that  in  which  all  the  66  items  were  used,  except  for  a  few 
items.  Table  4  shows  the  comparison  of  these  two  common 
f  actors . 
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Table  2 


Variance  in  the  Seven  Factor  Space  Accounted 
for  by  Each  Factor  following  Varimax  Rotation 


F  actor 

Variance 

%  of  variance 

Cumulative  %  of 
variance 

1 

10.3355 

41.1806 

41.1806 

2 

3.7101 

14.7825 

55.9632 

3 

2.6446 

10.5370 

66.5001 

4 

2.6218 

10.4463 

76.9464 

5 

2.5452 

10.1412 

87.0876 

6 

1.6264 

6.4801 

93.5677 

7 

1.6141 

6.4312 

99.9989 

Table  3 

The  Item  Grouping  by  the  Factors 

1(  >-40)  2 (  >.40)  3(  >.40)  4 (  >.40)  5(  >.40) 

3, 5, 6,9,  11,-23,  2, 4, 7, 8,  21,23,  1,15,10, 

12,13,16,  31,33,  10  25  42 

17,18,19,  34,36, 

22,29,30,  37 

33,35,41 , 

43,44,45, 

2  ( 1 1 1 ) 


Table  4 


The  Common  Factors  in  the  66-item  and  46-item  Analyses 


66-item  Analysis 

46-item  analysis 

! loadings ! > . 65 

3,5,6,9,13,17,18 

3,5,6,9,13,17,18 

19,30,35,43,44,) 

19,30,35,43,44, 

2 (III) 

2 ( II I ) 

.  65> ! loadings ! > . 40 

10,12,22,33,41, 

12,16,22,29,33, 

45 

41,45 

Item  10,  "The  equipment  made  it  difficult  to  concen¬ 
trate  on  the  course  material,"  was  dropped  in  the  46-item 
analysis  but  new  items,  16  and  29,  came  in  there.  Items  16 
and  29  were  "Material  which  is  otherwise  interesting  can  be 
boring  when  presented  on  PLATO,"  and  "I  was  aware  of  efforts 


F  actors 
Items 
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to  suit  the  material  specifically  to  me,”  respectively. 

The  second  factor  in  the  46-item  analysis  had  almost 
the  same  items  as  the  third  factor  in  the  66-item  analysis. 
The  students  felt  too  much  material  was  presented  and  could 
not  work  at  their  own  pace.  Also  they  felt  uncertain  about 
their  performance  and  the  situation  made  them  tense.  They 
complained  that  irrelevant  and  ambiguous  questions  were 
asked  in  the  lessons. 

The  third  factor  in  the  46-item  analysis  had  the  same 
items  as  the  fifth  factor  in  the  66-item  analysis,  except 
for  item  7,  "I  was  more  involved  in  operating  the  terminal 
than  in  understanding  the  course  material."  Items  that  fell 
in  the  factor  were  item  2,  "The  learning  was  too  mechani¬ 
cal";  item  4,  "The  equipment  interfered  with  learning";  and 
item  8,  "No  one  cared  if  I  learned  or  not,  so  I  felt  isolated 
and  alone." 

The  fourth  factor  in  the  46-item  analysis  shared  some 
items  with  the  first,  second,  and  third  factors  in  the  same 
46-item  analysis  and  it  is  very  difficult  to  give  an  ade¬ 
quate  interpretation  of  it. 

The  reason  that  we  reanalyzed  the  original  66-item 
analysis  presented  in  "Chanute  Report"  by  reducing  the 
number  of  items  to  46  was  that  the  first  factor  (common 
factor)  had  too  many  items  clustered  together  and  the  other 
factors  were  of  only  few  items.  Moreover,  we  felt  the  common 
factor  had  to  be  separated  out  into  two  groups,  those  having 
higher  loadings  and  lower  loadings;  in  other  words,  the  type 
I  and  II  as  we  mentioned  before.  Thus  the  46-item  analysis 
did  not  give  us  satisfactory  results,  so  we  separated  the  46 
items  into  2  subsets  of  items:  one  containing  the  items 
which  expressed  a  favorable  attitude  toward  PLATO  and  the 
other  containing  the  items  which  expressed  a  negative  atti¬ 
tude  toward  PLATO.  The  26  items  having  loadings  greater  than 
or  equal  to  .3  in  absolute  value  on  the  first  factor  were 
the  subset  of  favorable  items  and  the  29  items  having  factor 
loadings  greater  than  or  equal  to  .3  in  absolute  value  on 
the  factors  other  than  the  first  one  were  the  second  subset 
of  items.  These  items  in  the  two  subsets  were  as  shown 
below. 


The  list  of  items  analyzed  separately. 


Subset  of  26  items 


3,5,6,7,9,10,12,13,14,16,17,18,19,22,24 

25,29,30,33,35,39,41,43,44,45,46 


Subset  of  29  items  1,2,4,7,8,10,11,12,14,15,16,20,21,23,24 

25,26,27,28,29,31,32,34,36,37,38,39,40, 

42. 
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The  26-item  subset.  We  analyzed  the  two  subsets  sepa- 
ratelYT  At  first,  a  principal  components  analysis  was  done 
on  the  26  item  subset  and  the  resulting  factor  matrix  was 
rotated  by  the  varimax  method.  The  first  factor  was  of 
items  having  loadings  with  absolute  values  greater  than  or 
equal  to  .65  in  Table  4  except  for  items  35  and  2  ( 1 1 1 ) .  This 
factor  was  named  FI  and  retained  for  later  analysis.  The 
items  having  lower  loadings  in  the  46-item  analysis  which 
were  shown  in  Table  4  (12,  16,  22,  29,  33,  ^1>  45)formed  the 
second  factor  in  this  analysis  and  it  was  named  FII.  FII 
became  one  of  the  discriminators  of  students'  performance  as 
is  shown  later  in  this  chapter.  The  newly  extracted  third 
and  fourth  factors  in  this  analysis  were  ignored  and  not 
used  in  the  further  analysis. 

The  29-item  subset  was  analyzed  using  the  same  proce¬ 
dure  as  with  the  26-item  subset.  The  resulting  first  factor 
was  of  the  items  in  the  second  factor  of  46-item  analysis 
(11,  23,  31,  34,  36,  37)  and  of  the  fourth  factor  of  the  same 
analysis  (21,  23,  and  25)  together.  The  second  factor  was  of 
the  items  in  the  fifth  factor  of  the  46-item  analysis  (1,  5, 
40)  in  Table  3.  The  third  factor  included  the  items  in  the 
factor  3  in  Table  3  (2,  4,  7,  8,  10).  These  three  new  factors 
were  retained  in  later  analyses  and  were  designated  Fill, 

FIV,  and  FV  in  the  report. 

A  summary  of  the  final  factors  is  given  in  Table  5. 


Table  5 

Eight  Factors  Extracted  From 
the  Student  Attitude  Questionnaire 


FACTOR  1 

Personal  Involvement  with  PLATO,  or 
Acceptabil ity 

3.  I  felt  challenged  to  do  my  best  work. 

5.  I  felt  as  if  someone  were  engaged  in 
conversation  with  me. 

6,  As  a  result  of  having  studied  by  this 
method,  I  am  interested  in  learning  more 
about  the  subject  matter. 


9. 


I  felt  as  if  I  had  a  private  tutor. 


.67 
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13.  My  feeling  toward  the  course  material  after 
I  had  completed  the  PLATO  portion  of  the 
course  was  favorable.  .66 

17.  I  was  satisfied  with  what  l  learned  while 

taking  the  course.  .62 

18.  In  view  of  the  amount  I  learned,  this  method 
seems  superior  to  classroom  instruction  for 

many  courses.  .75 

19.  I  would  prefer  PLATO  to  traditional  instruction  .73 

30.  I  was  encourged  by  the  responses  given  to  my 

answers  of  questions.  .58 

43.  The  lessons  on  PLATO  were  interesting  and 

really  kept  me  involved.  .53 

44.  What  I  learned  from  PLATO  made  the  classroom  and 

Laboratory  instruction  easier  to  understand.  .68 

FACTOR  II 

Perception  of  effectiveness  of  PLATO 

12.  PLATO,  as  used  in  this  course,  is  an  inefficient  -.51 
use  of  the  student's  time. 

22.  The  responses  to  my  answers  seemed  appropriate.  .49 

33*  I  felt  I  could  work  at  my  own  pace.  .66 

35.  Material  which  is  otherwise  boring  can  be  .61 

interesting  when  presented  by  PLATO. 

41.  Computer-assisted  instruction  did  not  make  it  -.60 

possible  for  me  to  learn  quickly. 

45.  The  PLATO  lessons  were  dull  and  difficult  to  -.57 

follow. 

2.  As  a  student,  I  do  my  best  work  with  .60 

PLATO  lessons. 
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FACTOR  III 

Alienation  -  PLATO  as  a  task,  not  learning 

11.  The  situation  made  me  quite  tense.  .56 

14.  I  felt  frustrated  by  the  situation.  .50 

20.  PLATO  instruction  is  just  another  step  toward  .42 

de-personalized  instruction. 

21.  I  was  concerned  that  I  might  not  be  .49 

understanding  the  material. 

23.  I  felt  uncertain  as  to  my  performance  in  the  .59 

programmed  course  relative  to  the  performance 
of  others. 

25.  I  found  myself  just  trying  to  get  through  the  .48 

material  rather  than  trying  to  learn. 

28.  i  guessed  at  the  answers  to  some  questions.  .52 

31.  In  view  of  the  time  allowed  for  learning,  I  .67 

felt  too  much  material  was  presented. 

34.  Questions  were  asked  which  I  felt  were  not  .57 

related  to  the  material  presented. 

36.  I  could  have  learned  more  if  I  hadn't  felt  .64 

pushed . 

37.  I  was  given  answers  but  still  did  not  understand  .58 
the  questions. 


FACTOR  IV 
Anti  System 

1.  The  method  by  which  I  was  told  whether  I  had  .65 

given  a  right  or  wrong  answer  Became  monotonous. 

15.  I  found  the  computer-assisted  instruction  .61 

approach  in  this  course  to  be  inflexible. 


40.  While  on  PLATO,  I  encountered  mechanical 
malfunctions. 


.60 
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FACTOR  V 
Impersonal  PLATO 

2.  Nobody  really  cared  whether  I  learned  .56 

the  course  material  or  not. 

4.  I  felt  isolated  and  alone.  .43 

7.  I  was  more  involved  in  operating  the  terminal  .55 
than  in  understanding  the  course  material. 

10.  The  equipment  made  it  difficult  to  concentrate  .50 
on  the  course  material. 

16.  Material  which  is  otherwise  interesting  can  be  .47 
boring  when  presented  on  PLATO. 

24.  I  was  not  concerned  when  I  missed  a  question  .53 

because  nobody  was  watching  me. 

29.  I  was  aware  of  efforts  to  suit  the  material  -.55 

specifically  to  roe. 

38.  The  course  material  was  presented  too  slowly.  .57 


FACTOR  VI 

Positive  feeling  toward  PLATO 
Working  with  the  PLATO  system: 


(III) 

7.  Is  fun.  .76 
9.  Is  challenging.  .85 
12.  Makes  me  proud  of  myself.  .83 
16.  Is  enjoyable.  .70 
18.  Is  exciting. 


.73 
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FACTOR  VII 


Negative  feeling  toward  PLATO 

Working  with  the  PLATO  system: 

8. 

Is  frustrating. 

.83 

10. 

Is  annoying. 

.89 

11. 

Is  confusing. 

.78 

13. 

Is  boring. 

.52 

III.  MEASURING  THE  GAIN  SCORES 
Kikumi  Tatsuoka 

The  achievement  data  collected  for  the  evaluation  study 
of  Chanute  AFB  CBE  project  suffered  from  low  reliabilities 
of  the  tests,  hence  there  is  a  strong  possibility  that  the 
error  of  measurement  on  the  test  scores  may  wash  away  the 
correlations  of  the  attitude  scores  with  the  achievement 
scores  if  we  use  the  observed  scores  that  we  have  now.  In 
order  to  avoid  the  problem  of  error  of  measurements,  we  will 
use  a  special  technique  that  has  been  developed  recently. 
This  problem  was  called  to  the  attention  of  educational 
measurement  specialists  in  connection  with  measuring  the 
gain  scores  of  a  posttest  from  a  pretest. 

Measuring  gain  scores  has  been  used  in  research  in 
education  and  psychology  even  though  it  has  some  consider¬ 
able  problems.  In  many  situations  such  as  evaluation  of 
educational  programs,  gain  scores  appear  to  be  the  natural 
measure  to  be  looked  into. 

There  were  three  major  persisting  dilemmas:  "low 
reliability  and  error  of  measurement,"  "regression  effect," 
and  "over-correction,  under-correction."  Ever  since  E.  L. 
Thorndike  0924)  pointed  out  these  dilemmas,  measurement 
specialists  in  education  and  psychology  have  tried  to  over¬ 
come  the  difficulties  by  suggesting  various  methods  to  esti¬ 
mate  gain  scores.  He  looked  into  the  reliability  of  the 
difference  between  scores  of  two  tests,  and  showed  that  this 
reliability  is  lower  than  the  reliabilities  of  two  tests 
taken  separately.  This  fact  was  explained  in  that  errors  of 
measurement  in  both  separate  tests  affect  the  difference 
score  and  whatever  was  common  to  both  measures  was  cancelled 
out  in  the  difference.  In  our  study,  coefficient  alpha  of 
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pretest  and  posttest  I  were  -3960  and  .6300  respectively. 

But  the  reliability  of  simple  difference  scores  was  only 
.1047,  which  was  smaller  than  the  reliability  of  either 
pretest  or  posttest  I. 

Cronbach  and  Furby  (1970)  suggested  the  use  of  multiple 
regression,  that  is,  to  regress  the  true  gain  score  on  the 
space  spanned  by  both  tests.  By  adopting  a  multiple  regres¬ 
sion  approach,  they  succeeded  in  avoiding  the  persistant 
problem  of  whether  the  two  tests  measured  the  same  thing. 
Indeed,  estimating  the  true  gain  score  by  the  multiple 
regression  method  provided  very  efficient  gain  scores. 

The  simple  difference  score  typically  has  a  negative 
correlation  with  the  pretest.  Hence  it  implies  that  if 
individuals  with  high  gain  scores  are  to  be  selected,  there 
will  be  an  over-representation  of  people  with  low  pre-test 
scores  as  an  artifact  due  to  the  negative  correlation 
between  gains  and  pretest  scores.  The  method  of  using 
regression  gain  scores  (or  residual  scores)  was  introduced 
by  Debois  (1957)  and  Manning  (1962)  in  order  to  avoid  this 
dilemma.  Regression  gain  has  a  zero  correlation  with  the 
pretest.  This  gain  score  was  obtained  by  subtracting  the 
predicted  pretest  score  from  the  corresponding  posttest 
score.  We  calculated  the  gain  scores  by  regression  method 
also.  The  resulting  gain  has  zero  correlation  with  the 
pretest  and  .86  with  posttest  I.  As  indicated  by  O'Connor 
(1972)  and  Linn  and  Slinde  (1977),  the  reliability  of 
regression  gain  is  as  low  as  that  of  raw  difference  scores. 
Our  regression  gain  has  a  reliability  of  .33  which  is  higher 
than  the  reliability  of  difference  scores.  But  it  is  still 
very  low,  and  therefore  it  is  risky  to  make  any  decisions 
about  individuals  on  the  basis  of  gains  from  pre-  to 
posttesting  periods  on  the  basis  of  this  regression  gain 
score.  It  is  possible  to  observe  some  individuals  with 
large  difference  scores,  even  without  any  real  change. 

Knowing  that  the  reliability  of  our  regression  gain  was  only 
.33*  we  investigated  the  relationships  between  performance 
scores  and  attitude  scores  and  compared  the  results  with 
those  we  obtained  using  the  multiple  regression  method. 
Applying  this  method,  we  obtained  gain  scores  having  a 
reliability  of  .74. 

The  calculation  process  of  estimating  an  individual  true 
gain  score  by  the  multiple  regression  method  that  was  intro- 
duced  by  Cronbach  and  Furby  (1970)  was  tedious  and  compli¬ 
cated.  If  other  measures  than  pretest  and  posttest  were  used 
to  step  up  the  multiple  R,  then  the  procedure  would  be  very 
messy  algebraical ly  and  numerically.  A  mathematically 
simple  method  of  regressing  true  gain  scores  on  several 
variables  was  developed  by  the  first  author  of  this  report 
(Tatsuoka,  1975).  The  procedure  made  it  possible  to  separate 
the  reliability  (squared  multiple  R)  of  the  estimated  true 
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gain  into  the  increment  of  R2  provided  by  each  variable. 

The  reliability  of  the  difference  score  xp-x.,  with  x-|  by 
posttest  I  and  X£  by  pretest,  would  be  calculated  first, 
then  the  additional  contribution  by  regressing  t2~t-j  (true 
gain  score)  onto  xi  and  X2  would  be  calculated.  The  sum  of 
both  values  would  be  the  reliability  or  squared  multiple  R 
of  the  estimated  true  gain  scores  by  the  multiple  regression 
method.  Table  6  gives  the  result  of  numerical  calculation 
applied  to  one  pretest  and  posttest  I.  The  estimated  gain 
score  of  t2~ti,  obtained  by  regressing  it  on  x-|  and  X2  will 
be  given  by  R  ( t2_t  1 !  x ,  x  p ) .  The  estimated  gain  score  of 
t2"ti>  using  x^,  x2  and  the  scores  of  Block  test  2  is  given 
by  R C 1 2”fc 1  1  x1,x2>  B1^»  etc. 


Table  6 


The  Estimated  True  Gain  Scores  Obtained  by 
Multiple  Regression  Method 


Variables 


Incremental  R2 


Reliability  (R2) 


X?“X  1 

R(t2-t1 

R  (tp-t-, 
R(tp-t] 


R(tx-t1 


X1  ,x2 

x}jx|;bi  ^ 

x 1 i x  2 1 B2  > ) 


x2 » §2} 


2,b3) 


— 

- 

.  1047 

63578 

.74048 

058 

94 

.79942 

1 16 

50 

.91592 

02186 

.93778 

The  number  of  students  is  110,  and  the  reliabilities  of  the 
pretest  and  posttest  I  are  .3960  and  .6300  respectively. 


The  reliability  of  the  difference  score  was  only 
0.10470  but  the  multiple  regression  gain  increased  to  .63578 
which  is  six  times  as  much  as  that  of  the  difference  score. 
We  use  a  notation  of  F(t2~ti  !  X2_x^  )  for  the  regression  of 
true  difference  scores  t?-tf  onto  the  difference  scores  of 
pretest  scores  from  posttestl,  where  Xp  stands  for  posttest 
II  and  x-j  for  pretest.  R  ( t2-t  f !  X2 » x  -j ,  Bp)  was  the  true  gain 
that  was  regressed  on  pretest,  posttest2  and  block  exam  2, 
B2.  The  increment  of  squared  multiple  R  by  adding  block 
exam  2  to  predict  true  gain  score  in  R  ( t2_t -j '  X2 » x -j )  was 
.05894.  By  adding  block-tests,  the  increment  of  .71650  in 
squared  multiple  R  was  calculated.  Block  exam  4  had  an 
increment  of  .02186.  The  rate  of  the  increment  by  block 
exam  4  suggested  that  the  PLATO  lessons  in  block  3  contri¬ 
buted  most  significantly  to  the  gain  measurement  by  the 
special  topical  test  given  at  the  beginning  and  end  of  the 
program . 

Table  7  shows  a  comparison  of  traditional  regression 
gain  and  multiple  regression  gain  using  the  pretest,  post¬ 
test  and  Block  tests  2,3  and  4,  8  (t2"t  ■)  !  x  1 ,  X2 »  B2  >  B^  >  ) . 
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They  are  expressed  as  A  and  B,  respectively,  in  the 
following  table. 


Table  7 

The  Correlations  of  the  Gains  with  Various  Tests 


Test 

A 

B 

pretest 

.0 

.6820 

posttest  I 

.8643! 

.8354 

posttest  II 

.3594* 

.4547 

final  grade 

.2677 

.4841 

block  1 

.1293 

.381 1 

block2 

.  1842 

.1603 

block3 

.3031 

.6047 

block4 

.  1595 

.3422 

p< .  05 

It  is  interesting  to  note  that  the  expanded  multiple 
regression  gain  had  larger  correlations  with  posttest  II, 
final  grade,  and  block  1,  3,  4  tests  than  the  correlations  of 
the  traditional  regression  gain  with  these  tests.  Since  the 
expanded  multiple  regression  gain  had  a  reliability  of 
.9378,  the  estimated  gain  scores  were  very  close  to  the  true 
gain,  tp-t-j.  It  is  reasonable  to  assume  that  block-tests 
would  affect  the  gains  because  the  gain  should  be  a  result 
of  learning  throughout  the  program,  and  block  exams  were  the 
measures  of  each  learning  segment  of  this  period.  The 
correlations  of  the  regression  gain  with  block  tests  were 
not  significant  except  for  that  of  block  3>  which  was  .3031, 
while  the  expanded  multiple  regression  gain  had  correlations 
of  .6047  with  block  3»  .3811  with  block  1,  and  .3422  with 
block  4.  However,  the  correlations  of  both  of  the  gains 
with  pretest  and  posttest  1  were  contradictory  to  the 
expectation  that  gain  should  have  zero  correlation  with  a 
pretest.  The  regression  gain  had  zero  correlation  with  the 
pretest  but  both  of  the  expanded  multiple  regression  gain 
and  multiple  regression  gain  did  not  have  it.  Indeed,  it 
is  mathematically  impossible  to  expect  that  a  true  gain 
vector  tp-t-j  and  pretest  vector  would  be  orthogonal. 


IV.  RELATIONSHIP  BETWEEN  PERFORMANCE  AND  ATTITUDE 

Five  items  in  the  first  section  of  the  attitude 
questionnaire  were  repeatedly  given  to  the  students  (four 
times  at  the  end  of  each  block).  The  results  of  repeated 
measures  analysis  of  variance  are  given  later. 
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The  item  in  section  II  and  the  second  item  in  section 
III  were  rotated  by  factor  analysis  and  five  factors  were 
extracted.  These  five  factors  were  used  to  relate  stu¬ 
dents'  performance  scores  together  with  the  three  factors 
obtained  from  items  7  through  18  in  section  III.  Items  1, 

3,  4,  5,  6  and  19,  20,  21  in  section  III  were  also  used  in 
order  to  relate  with  the  performance  score  and  the  factors 
FI  through  FVII. 

Seven  factors  were  correlated  with  the  scores  of  the 
four  block  exams  and  the  time  that  a  student  needed  to  com¬ 
plete  a  block.  It  turned  out  that  the  factors  did  not 
correlate  with  the  time  data. 

The  relation  between  time  in  block  and  mechanical  failure. 
Question  20  in  Section  III  was  "How  often  has  PLATO  worked 
when  you  have  attempted  to  use  it?"  The  responses 
correlated  with  time  in  blocks  1  and  4  at  -.25  and  -.27 
respectively,  but  at  -.14  and  -.03  with  blocks  2  and  3- 
Apparently,  there  was  a  considerable  amount  of  system  fail¬ 
ures  during  blocks  1  and  4,  and  thus  the  mechanical  trouble 
affected  students'  study  time.  The  more  often  PLATO 
encountered  mechanical  trouble,  the  more  time  students  had 
to  spend  to  complete  the  instruction.  The  interruptions 
made  students'  learning  less  efficient.  The  time  in  block  4 
correlated  with  question  21  at  -.26.  Since  this  question 
asked  "During  your  school  day,  if  you  had  your  choice,  how 
much  of  your  time  would  you  spend  working  with  PLATO?"  the 
time  needed  for  completing  block  4  had  a  non-significant 
relationship  with  the  motivation  that  was  asked  in  question 
21.  It  is  interesting  to  note  that  time  needed  to  complete 
block  4  correlated  with  questions  19,  20,  and  21  at  .27, 

-.05,  and  -.26.  Time  in  block  4  was  affected  by  mechanical 
interruptions  but  this  did  not  make  the  students  want  to 
stop  using  PLATO.  However,  it  did  affect  their  wishes  to 
spend  time  working  with  PLATO  in  a  negative  direction.  The 
summary  of  correlation  between  time  in  blocks  and  mechanical 
failure  is  given  in  Table  8. 


Table  8 


Time  in  Blocks  and  Mechanical  Failure 


Question 

block  1 

Block2 

Block3 

Block4 

19 

-.25* 

-.  14 

-.03 

-.27* 

20 

13 

-.20 

-.  10 

-•05 

21 

-.  1 1 

-.02 

-.  10 

-.26* 

*p< . 05 
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The  relation  between 
failure.  The  most  interest 
not  mechanical  interruptio 
From  our  study,  there  was 
effect.  But  it  should  be 
this  study  was  controlled 
Hence  students  were  forced 
criterion-referenced  tests 
lesson . 


performance  and  mechanical 
ing  question  would  be  whether  or 
n  affected  students'  performance, 
no  indication  of  a  negative 
noted  that  students'  learnning  in 
by  a  mastery  learning  strategy, 
to  study  until  they  passed  the 
given  at  the  end  of  each  PLATO 


Table  9 

Correlation  of  Performance  and  Questions  19,  20,  21 


Ques¬ 

tion 

Gain 

Pre¬ 

test 

Post- 

testl 

Post- 

testll 

Final 

Block 

1 

Block 

2 

Block 

3 

Block 

4 

19 

0.  18 

.  17 

.20 

-.04 

.  17 

.13 

.15 

.08 

.  15 

2C 

0. 17 

.05 

.18 

.10 

.  16 

.22 

.09 

.20 

.13 

21 

0.06 

-.  10 

.00 

.01 

-.07 

.04 

-.04 

.01 

-.07 

The  relation  between  attitude  factors  and  mechanical 
failure.  Students'  attitudes  were  affected  by  mechanical 
interruptions  in  a  negative  way.  Both  FI  and  FII  correlated 
with  19  at  .49  and  with  20  at  -.45,  -.43  respectively. 

Factors  FVI ,  FVII  also  correlated  with  19  at  .47,  -32  and 
with  20  at  -.42  and  .39.  Negative  attitude  factors  Fill, 

FIV,  FV — alienation,  frustration  or  training  stress  as  Fill, 
anti-system  attitudes  as  FIV,  and  impersonal  PLATO  as  FV — 
did  not  correlate  with  19,  but  Fill  and  FIV  correlated  with 
20  at  .27  and  .23.  Frequent  occurance  of  mechanical  inter¬ 
ruptions  made  students  less  favorable  toward  PLATO  and  PLATO 
lessons,  produced  more  frustration  in  learning,  and  made  the 
students  complain  more  about  impersonal  learning.  It  was 
interesting  to  note  that  FIV  correlated  with  20  at  .15  which 
is  not  significant.  Items  in  FIV  were  anti-system  attitude 
in  a  general  way,  so  this  just  might  imply  that  they  just  did 
not  like  the  system  a  priori.  Therefore  frequency  of  mechani¬ 
cal  interruption  did  not  affect  or  stimulate  their  anti¬ 
system  attitude.  No  matter  how  well  the  system  worked,  or 
how  frequently  the  system  failed,  this  attitude  must  have 
always  been  negative  toward  PLATO.  Question  21  correlated 
with  all  factors  in  the  same  direction  as  19  did.  The  high 
rate  of  mechanical  interruption  discouraged  students'  wishes 
to  work  with  PLATO. 
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Table  10 

The  Correlation  of  the  Attitude  Factors 
with  Mechanical  Failure 


FI 

FII 

Fill 

FIV 

FV 

FVI 

FVI  I 

19 

20 

21 

19 

.49 

.40  - 

.17  - 

.  10  - 

.  17 

.47 

-.32  1 

.00 

-.29 

.42 

20 

-.45 

-.43 

.27 

.  15 

.23 

-.42 

.39  - 

.29 

1.00 

-.41 

21 

.72 

.63  - 

•  34  - 

.29  - 

.47 

.70 

-.52 

.42 

-.41 

1.00 

The  relation  between  attitude  factors  and  achievement. 
Correlational  analysis  of  factors  and  performance  scores 
revealed  that  FII,  Fill,  and  FVI  were  discriminators  of 
performance. 


Table  11 

The  Correlation  of  the  Attitude  Factors 
with  Achievement 


Pre¬ 

Post- 

Final 

Reg . 

Mult . 

test 

testl 

grade 

gain 

gain 

FI 

•  °4 

.  18 

•  20 

.  18 

•  15. 

FII 

.24* 

-.291 

.28* 

.19 

.32* 

Fill 

-.33 

-.35* 

-.37* 

-.20 

.37* 

FIV 

•  03 

-.00 

-.03 

-.02 

-.01 

FV 

-.  06 

.  16 

-.  15 

-.  15 

-.  16 

FVI 

.  10 

.23* 

.27 

.21 

-.23* 

FVI  I 

-.  1  1 

-.  12 

-.  19 

-.08 

-.  12 

*p< . 05 


The  correlations  of  factor  FI  with  performance  scores 
were  not  significant.  Personal  involvement  with  PLATO  and 
favorable  attitude  toward  PLATO  did  not  relate  with  academic 
performance  measured  by  a  special  topical  test,  final  grade, 
and  gain  scores.  Table  11  does  not  include  the  correlations 
of  FI  with  block  exams  and  posttest  II,  but  they  were  all  non 
significant.  Factor  FIV  represented  the  anti-system 
atti tude--compl a  in ing  that  answering  to  responses  became 
monotonous,  the  CAI  approach  was  inflexible,  and  mechanical 
malfunctions  encountered  while  on  PLAT0--and  did  not  cor¬ 
relate  with  performances.  Attitudes  toward  a  teaching 
nied ium--whatever  it  was--were  independent  from  achievement 
of  learning,  and  those  attitudes  must  have  been  planted 
before  the  program  started.  Pretest  did  not  correlate  with 
FI,  FIV,  and  FV.  Factors  FIV  and  FV  stayed  at  zero  correla¬ 
tion  level  as  the  program  went  by,  while  correlation  of  FI 
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increased  according  to  an  order  of  tests  given  from  earlier 
to  later.  However,  these  values  were  not  statistically 
significant.  Hence  no  conclusions  should  be  from  Table  11. 

Factor  FV,  impersonal  PLATO,  did  not  correlate  with 
performance  scores.  Items  in  FV  represented  the  weakness  of 
instructions  that  had  been  attributed  to  CAI  in  general, 
using  a  machine  to  teach  and  interact  with  students.  "Nobody 
really  cared  whether  a  student  learned  the  course  material 
or  not";  "I  was  not  concerned  when  I  missed  a  question 
because  nobody  was  watching  me";  "I  felt  isolated  and 
alone";  "I  was  not  aware  of  efforts  to  suit  the  material 
specifically  to  me  and  I  thought  the  course  material  was 
presented  too  slowly."  Students  were  more  involved  in 
operating  the  terminal  than  in  understand ing  the  course 
material  and  hence  the  equipment  made  it  difficult  to 
concentrate  on  the  course  material.  The  question  of  to  what 
extent  mastery  learning  techniques  played  a  role  of  reducing 
the  effect  of  FV  on  the  students'  achievements  was  not 
clearly  answered  from  our  data. 


Table  12 


The  Correlations 

of 

Attitudinal 

Data 

and  FI, 

IV,  and 

Pre¬ 

test 

B1_ 

B2 

B3 

§1 

Post¬ 
test  1 

FI  .04 

.08 

-09 

.  12 

.  18 

.18 

FIV  .03 

-.07 

.01 

-.02 

-.05 

-.00 

FV  -.06 

-.04 

-.02 

-.08 

-.09 

.  16 

FI I  was  originally  clustered  together  with  FI  in  the 
first  analysis  we  conducted.  The  correlation  of  FI  +  FII, 
items  in  FI  and  FII  together,  and  FI  and  FII  separately  are 
presented  in  Table  13- 


Table  13 

The  Comparison  of  FI,  FII,  and  FI+FII 


Pre¬ 

Post¬ 

Final 

Reg . 

Mult. 

test 

test  I 

gr  ad  e 

gain 

gain 

FI 

.04 

.  18, 

.20 

.  18 

.  15 

FII 

.  24  *  * 

-.29 

.28* 

.19 

.32 

r  I 
FII 

+ 

•  23 

.  24** 

.20 

.22 
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FII  was  significantly  correlated  with  all  performance 
scores  except  for  regression  gain,  but  FI  +  FII  were  not  sig¬ 
nificant  except  for  the  final  grade.  FII  was  perception  of 
effectiveness  of  PLATO  lessons;  the  students  felt  they  could 
work  at  their  own  pace  and  thought  that  material  which  was 
otherwise  boring  could  be  interesting  when  presented  by 
PLATO.  Moreover,  they  thought  CAI  made  it  possible  for  them 
to  learn  quickly  and  did  their  best  work  with  PLATO  lessons. 
They  believed  that  PLATO  lessons  were  not  dull  and  not  dif¬ 
ficult  to  follow.  PLATO  as  used  in  this  course  was  an 
efficient  use  of  the  student’s  time  and  responses  to  their 
answer  seemed  appropriate.  It  was  interesting  to  note  that 
FII  correlated  with  the  performance  scores  but  FI  did  not. 

Fill  was  also  a  discriminator  of  performance  scores. 

This  factor  expressed  frustration  and  stress  in  learning 
PLATO  lessons.  The  factor  represented  "alienation,”  i.e. 
feeling  little  involvement  with  learning,  and  seeing  PLATO 
only  as  a  task  necessary  to  complete  the  program.  Items 
included  "PLATO  instruction  is  just  another  step  toward 
depersonalized  instruction";  "Too  much  material  was 
presented,  so  I  felt  pushed";  "I  was  uncertain  as  to  my 
performance  relative  to  the  performance  of  others."  "I 
could  not  follow  the  questions  I  have  been  asked  and  I  felt 
some  questions  were  not  related  to  the  material  presented"; 
hence,  "I  guessed  at  the  answers  to  some  questions." 

Students  felt  "Tense  and  frustrated  by  the  situation."  They 
were  concerned  that  they  "might  not  be  understanding  the 
material,"  and  "just  tried  to  get  through  the  material 
rather  than  trying  to  learn."  FII  correlated  positively  and 
Fill  correlated  negatively  with  all  performance  scores 
consistently. 

FVI  correlated  with  some  performance  scores,  posttestl, 
final  grade  and  multiple  gain  scores  but  their  magnitudes 
were  smaller  than  those  of  FII  and  Fill.  FVI  represented 
a  positive  feeling  toward  PLATO  teaching.  For  example, 
"Working  with  the  PLATO  system  was  challenging,  made 
me  proud  of  myself,  was  fun,  exciting  and  enjoyable." 

FVII  represented  the  negative  feeling  toward  PLATO 
teaching--that  working  with  the  PLATO  system  was  annoying, 
frustrating  and  confusing.  But  FVII  did  not  correlate  witn 
any  one  of  the  performance  scores. 

The  relation  of  Posttest  II  with  attitude  factors  and 
time  in  blocks~  Posttest  II  was  given  at  the  training 
course.  The  test  consisted  of  the  same  items  as  the  pretest 
and  posttest  I  that  was  fiven  at  the  end  of  the  PLATO  program, 
but  orders  of  items  were  randomly  changed.  The  time  needed 
for  completing  each  block  did  not  correlate  with  either 
pretest  or  posttest  I,  but  the  time  snent  in  blocks  3  and  A 
correlated  with  posttest  II  at  -.Al  and  -.32  resoecti vely . 

The  scores  of  posttest  II  correlated  with  posttest  T,  final 


The  scores  of  posttestll  correlated  with  posttestl,  final, 

B 1 ,  B2 ,  B3,  and  B4  at  .482  ,  .440  ,  .292  ,  .365  ,  .425  ,  and 
.211  respectively  (  p<.01).  These  results  implied  that  time 
spent  in  the  last  two  blocks  had  more  effect  on  the  perform¬ 
ance  of  posttestll,  that  was  given  eight  weeks  after  the 
PLATO  program  was  over,  than  achievement  scores  of  blocks  3 
and  4  tests.  It  probably  was  related  to  a  retention-of- 
learning  effect. 

The  relation  to  attitude  factors  was  checked  and  only 
Fill  had  a  significant  correlation  with  posttest  II  of  -.302. 
The  correlation  of  FII  dronoed  to  .202  which  was  not 
significant  anymore.  It  indicated  that  Fill  was  also  a 
discriminator  of  future  performance. 


V.  SHORT  FORM  ATTITUDE  QUESTIONNAIRE 
Patrick  Maritz 

Parallel,  five-item  attitude  questionnaires  were 
repeatedly  administered  to  the  three  trainee  groups  at  the 
end  of  each  block  of  lessons.  The  questionnaire  was  designed 
to  measure  attitudes  towards  the  trainees'  respective  forms  of 
instruction.  Thus,  the  short  form  attitude  questionnaire 
was  able  to  measure  the  amount  and  direction  of  change  of 
trainees'  attitudes  towards  their  curriculum  during  the  four 
instructional  periods. 

Since  the  trainees  had  recently  arrived  at  their 
training  school  and  were  about  to  embark  upon  their  special¬ 
ized  fields  of  military  interest,  it  would  be  expected  that 
trainee  attitudes  would  initially  be  highly  positive  toward 
their  training  irrespective  of  the  form  of  their  training. 

As  the  trainees  experience  frustrations  encountered  during 
the  training,  the  average  group  attitudes  toward  their 
training  would  decline.  The  attitude  decline  would  become 
less  from  period  to  period  as  the  trainees  began  to  cope 
with  the  frustrations.  With  the  forms  of  frustration  during 
training  being  different  for  each  of  the  groups  and  attitude 
parameters  largely  being  a  function  of  the  effects  of 
instructional  frustration,  it  would  be  expected  that  the 
groups  would  have  different  initial  attitudes  and  different 
rates  of  attitude  change.  Type  one  training  was  a  pure 
c omputer- assi sted  instruction  (CAI)  curriculum.  Type  three 
was  a  programmed  text  instruction  presentation.  Several 
ratios  of  the  two  previous  types  make  up  the  second 
curriculum  form.  All  the  curricula  were  measured  with 
parallel  attitude  questionnaires  concerned  with  the  same 
aspects  of  curriculum.  The  aspects  considered  were 
enjoyment  of  the  time  spent  in  instruction,  perceived 
achievement  of  curriculum  goals,  desire  to  continue  with 
instructional  form,  general  sense  of  accomplishment  derived 


from  the  curriculum,  and  whether  the  curriculum  was 
challenging . 


Analysis  and  result.  A  completely  crossed  3x^x5 
factorial  analysis  of  variance  was  performed  upon  the  corres 
ponding  factors;  instructional  format,  blocks  of  time,  and 
questionnaire  items  concerned  with  attitudes  toward  curricu¬ 
lum.  The  students  were  nested  within  the  instructional 
format  and  crossed  with  blocks  and  items.  A  significance 
level  of  «r . 05  was  utilized.  For  the  main  effects,  all  of 
the  F-ratios  were  significant.  All  of  the  two-way  interac¬ 
tions  were  also  significant.  The  three-way  interaction  was 
not  significant. 

A  series  of  post-hoc  pair-wise  mean  contrasts  were 
performed  using  the  Sheff*  test  at  a  significance  level  of 
ei=.05.  The  Sheff*  test  was  used  due  to  unequal  cell  sizes. 
Pair-wise  contrast  for  instructional  formats  showed  that  the 
significantly  different  pair  of  means  was  the  mixed  form 
and  the  programmed  text  form.  For  blocks  of  time,  all  of 
the  pair-wise  differences  among  the  four  blocks  were  signi¬ 
ficant  with  the  exception  of  the  blocks  two-three  contrast. 
The  items  dealing  with  the  student's  attitudes  toward  curric 
ulum  had  no  differences  among  any  pair  of  means. 

Complex  contrasts  for  the  main  effect  of  instructional 
form  showed  a  significant  difference  between  the  mixed 
group's  mean  and  the  composite  mean  of  the  CAI  group  and  the 
programed  text  group.  Similarly,  when  the  programed  text 
group  was  contrasted  against  the  composed  mean  of  the  other 
two  groups,  the  difference  was  again  significant.  Since  the 
only  non-significant  pair-wise  difference  for  blocks  of  time 
was  between  blocks  two  and  three,  the  only  complex  contrast 
performed  was  to  look  at  the  first  half  of  the  time  against 
the  second  half,  i.e.  blocks  one  and  two  against  three  and 
four.  The  blocks  complex  contrast  was  significant.  When 
the  greatest  pair-wise  difference  among  the  items  of  curric¬ 
ulum  was  not  at  the  significant  level,  it  was  indicated 
that  the  significant  ANOVA  F-ratio  for  the  item  main  effect 
was  reflecting  a  complex  contrast.  Therefore,  a  complex 
contrast  between  the  highest  mean  and  the  two  lowest  means 
was  performed  and  found  significant  thus  confirming  the 
F-ratio . 

The  results  show  that  initial  attitudes  toward  instruc¬ 
tion  among  the  groups  were  different  after  the  first  block 
of  lessons.  The  NP  groups  were  confronted  with  the  conven¬ 
tional  instructional  frustrations  of  lectures,  such  as 
failure  to  hear  or  comprehend  portions  of  the  lecture,  not 
being  able  to  immediately  repeat  difficult  points,  peer  dis¬ 
traction,  etc.  New  frustrations  of  machine  down  time, 
improperly  working  programs,  ambiguous  statements,  etc., 
were  the  problems  of  the  PB  groups.  The  PB  groups  may  have 
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had  as  many  frustrations  as  the  NP  groups  but  the  problems 
were  new  to  the  PB  group.  As  compared  to  the  familar  prob¬ 
lems  with  which  the  NP  group  coped,  the  new  PB  problems 
would  initially  have  a  lesser  attitude  depression  effect. 

The  CP  group’s  frustrations  were  partially  alleviated  by 
having  their  routines  broken  up  by  having  different  instruc¬ 
tional  settings.  One  type  of  frustration,  boredom,  was 
alleviated.  Thus,  the  groups  are  initially  ordered. 

The  next  question  is  whether  the  group’s  attitudes 
differentially  declined  during  the  periods.  With  the  excep¬ 
tion  of  the  NP  group  after  the  third  period  of  instruction, 
the  groups  displayed  a  smooth  negative  exponential  decline 
in  attitudes  even  though  remaining  favorable.  Being  con¬ 
fronted  with  familiar  frustrations  induces  the  NP  group's 
attitudes  to  decline  at  the  greatest  rate.  Being  presented 
with  new  roles  in  a  CAI  environment  mitigates  instructional 
frustrations  in  the  PB  groups  generating  the  most  stable 
attitudes  even  though  not  the  highest.  By  combining  the 
previous  two  methods,  the  CP  group's  attitudes  decline  at  a 
rate  between  the  other  two  types  while  maintaining  consis¬ 
tently  more  favorable  attitudes  than  the  previous  two 
groups.  The  groups  did  not  significantly  interact  with  the 
questionnaire  items  nor  the  items  with  the  blocks  of 
instruction.  Thus,  the  differences  attributable  to  the 
groups  are  mainly  a  function  of  attitude  differences. 

Conclusion.  Difference  among  the  groups  are  mainly 
attributable  to  attitude  differentials.  CP  produces  consis¬ 
tently  more  favorable  attitudes.  PB  attitudes  are  the  most 
stable.  Depending  on  how  much  weight  is  to  be  attributed  to 
degree  of  favorability  and  degree  of  stability  of  attitude, 
the  percent  of  instructional  time  on  PLATO  could  be 
determined.  A  more  extensive  investigation  of  the  percent 
of  time  spent  on  PLATO  would  help  to  determine  the 
interaction  between  attitude  favorability  and  stability. 


VI.  SUMMARY  AND  DISCUSSION 
Kikumi  Tatsuoka 

Factor  analysis  was  performed  on  the  66-item  attitude 
questionnaire  and  seven  factors  were  extracted.  They  are 
summarized  as  follows: 

I.  Personal  involvement  with  PLATO,  or  acceptability. 

II.  Perception  of  effectiveness  of  PLATO. 

III.  Alienation  -  PLATO  as  a  task,  not  learning. 

IV.  Anti-system. 

V.  Impersonal  PLATO. 

VI.  Positive  feeling  toward  PLATO. 

VII.  Negative  feeling  toward  PLATO. 
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Positive  perception  of  effectiveness  in  PLATO  lessons 
was  associated  with  better  achievement,  and  frustration  and 
stress  of  learning  led  to  less  favorable  achievement 
results.  It  probably  could  be  concluded  that  the  lessons  on 
the  PLATO  system  needed  to  implement  more  help  branches,  and 
more  individualized  feedback  on  the  questions.  The  length 
of  each  lesson  must  be  carefully  planned  so  that 
presentation  of  too  much  material  at  a  time  can  be  avoided. 
It  was  apparent  that  some  students  could  take  long  lessons 
that  might  continue  for  more  than  one  hour  or  two,  but  other 
students  felt  pushed  and  that  they  were  presented  with  too 
much  material.  Hence,  they  needed  more  time  to  complete 
their  assignment. 

It  was  interesting  to  note  that  regression  gain  did  not 
correlate  with  any  attitude  factors,  while  multiple  regres¬ 
sion  gain  and  pre-  and  posttest  I  correlated  with  some 
factors.  Although  the  reliability  of  regression  gain  scores 
was  low,  pre-  and  posttest  I  suffered  from  low  reliabilities 
too.  Moreover,  the  multiple  regression  gain  correlated  with 
three  Block  tests,  1,  3,  and  4,  at  moderately  high  correla¬ 
tion  values,  but  the  regression  gain  correlated  only  with 
block  3  test  at  .30,  which  is  barely  significant.  Final 
grade  and  posttest  II  showed  larger  correlation  values  with 
the  multiple  regression  gain  than  with  the  regression  gain. 
This  fact  implied  that  the  traditionally  used  regression 
gain  measured  something  different  from  all  other  tests. 
Therefore,  there  was  a  serious  doubt  about  using  regression 
gain  (residual  gain)  to  measure  the  difference  between  a 
posttest  and  a  pretest. 

The  gain  scores  measured  by  multiple  regression  with 
some  block  tests  were  also  affected  by  attitude  factors  FII, 
Fill,  and  FVI.  More  positive  attitude,  as  measured  by  FII 
and  FVI,  was  associated  with  a  better  gain,  while  more 
negative  attitude,  as  measured  by  Fill,  went  along  with  less 
gain  in  performance.  It  should  be  noted  that  the  achieve¬ 
ment  was  not  affected  by  mechanical  failure  or  system 
failure,  counter  to  the  instructors'  negative  expectation, 
but  students'  attitudes  were  affected  significantly  nega¬ 
tively.  The  interpretation  of  the  result  may  be  attributed 
to  the  mastery  learning  strategy  which  the  Chanute  AFB  CBE 
project  adopted  in  their  program,  and  students  were  forced 
to  study  until  they  passed  the  criterion-referenced  tests 
given  at  the  end  of  each  PLATO  lesson. 

The  time  spent  in  the  last  two  blocks  had  more  effect 
on  the  performance  of  posttest  II,  which  was  given  eight 
weeks  after  the  PLATO  program  was  over,  than  the  achievement 
scores  on  Block  tests  3  and  4  had.  This  is  particularly 
interesting  because  the  retention  may  depend  more  on  how 
much  time  a  student  spent  on  a  given  topic  until  he/she 
masters  it,  and  less  on  how  well  he/she  did  on  the  test. 
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The  analysis  of  the  short  form  attitude  questionnaire 
repeatedly  given  four  times  at  the  end  of  Block  tests 
revealed  that  both  PLATO-Based  and  Combined  PLATO  groups 
maintained  a  consistently  favorable  and  stable  attitude 
toward  the  program,  but  the  attitude  of  Non-PLATO  group 
fluctuated  according  to  their  instructors'  popularity. 
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BLOCK  # 

SHRED  ABC  -1 


PLATO  QUESTIONNAIRE 

NAME  SSAIJ  DATE 


Circle  the  number  that  represents  how  you  feel  about  the  following 
statements. 


1.  I  enjoyed  the  time 

2.  I  learned  what  the 
to  teach. 


I  spent  on  PLATO. 
PLATO  lessons  tried 


Very  Very 

Strongly  Strongly 
Disagree _ Agree 

0123456789 

0123456789 

0123456789 

01  23456789 
0  123456789 

01  23456789 


3.  I  would  like  to  study  more  PLATO 
lessons . 

4.  I  feel  PLATO  didn't  teach  me  very  much. 

5.  I  feel  challenged  to  do  my  best  work  on 
PLATO, 

6.  Wnat  have  you  enjoyed  most  about  using 
PLATO?  (You  may  use  the  other  side.) 


If  you  could  change  anything  about  PLATO  what  would  you  change? 
(You  may  use  the  other  side). 


fUCUIB  PAOK  BLAML-NC *  FILMED 


PLATO  QUESTIONNAIRE 


SECTION  II 

Very  Very 

Strongly  Strongly 
Disagree _ Agree 

1.  The  method  by  which  I  was  told  wnether  0123456789 

I  had  give  a  right  or  wrong  answer 

became  monotonous. 

2.  Nobody  really  cared  whether  I  had  0123456789 

learned  the  course  material  or  not. 

3.  I  felt  challenged  to  do  my  best  work.  01  23456789 

4.  I  felt  isolated  and  alone.  0123456789 

5.  I  felt  as  if  someone  were  engaged  in  0123456789 

conversation  with  me. 

6.  As  a  result  of  having  studied  by  this  0123456789 

method,  I  am  interested  in  learning 

more  about  the  suoject  matter. 

7.  I  was  more  involved  in  operating  the  0123456789 

terminal  than  in  understanding  the 

course  material. 

3.  The  learning  was  too  mechanical.  0123456789 

9.  I  felt  as  if  I  had  a  private  tutor.  0123456789 

10.  The  equipment  made  it  difficult  to  0123456789 

concentrate  on  the  course  material. 

11.  The  situation  made  rne  quite  tense.  01  23456789 

12.  PLATO  as  used  in  this  course,  is  an  0123456789 

inefficient  use  of  the  student's  time. 

13.  My  feeling  toward  the  course  material  0  1  23456739 

after  I  had  had  completed  the  PLATO 

portion  of  the  course  was  favorable. 

14.  I  felt  frustrated  by  the  situation.  01  23456789 

15.  I  found  the  computer- as  si sted  01  23456789 

instruction  approach  in  in  this  course 

to  be  inflexible. 

16.  Material  which  is  otherwise  interesting  0123456789 

can  be  boring  when  presented  on  PLATO. 
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Very  Very 

Strongly  Strongly 
Pi  sagree _ Agree 

17.  I  was  satisfied  with  what  I  learned  0  1  234567  89 

while  taking  the  course. 

18.  In  view  of  the  amount  I  learned,  this  0123456789 
method  seems  superior  to  classroom 

instruction  for  many  courses. 

19.  I  would  prefer  PLATO  to  traditional  01  23456789 

instruction . 

20.  PLATO  instruction  is  just  another  step  0123456739 
toward  de-personal ized  instruction. 

21.  I  was  concerned  that  I  might  not  be  0123456789 

understanding  the  material. 

22.  The  responses  to  my  answers  seemed  0123456739 

appropriate . 


2j.  I  felt  uncertain  as  to  my  performance  01  23456789 
in  the  programmed  course  relative  to 
the  performance  of  others. 

24.  I  was  not  concerned  when  I  missed  a  0123456789 

question  because  nobody  was  watching  me. 

25.  I  found  myself  just  trying  to  get  01  23456789 

through  the  material  rather  than 

trying  to  learn. 

26.  I  knew  whether  my  answer  was  right  or  0123456789 
wrong  before  I  was  told. 

27.  In  a  situation  where  I  am  trying  to  0123456789 

learn  something,  it  is  important  to  me 

to  know  where  I  stand  relative  to 
others . 

28.  I  guessed  at  the  answers  to  some  0123456789 

questions . 

29.  I  was  aware  of  efforts  to  suit  the  01  23456739 

material  specifically  to  me. 

30.  I  was  encouraged  by  the  responses  given  0123456789 
to  my  answers  of  questions. 

31.  In  view  of  the  time  allowed  for  0123456789 

learning,  I  felt  too  much  material  was 

presented . 
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Very  Very 

Strongly  Strongly 
Pi  sagree _ Agree 

32.  I  entered  wrong  answers  in  order  to  get  0123456789 
more  information  from  the  machine. 

33.  I  felt  I  could  work  at  my  own  pace.  01  23456789 

34.  Questions  were  asked  which  I  felt  were  0123456789 

not  relaed  to  the  materials  presented. 

35.  Material  which  is  otherwise  boring  can  0123456789 
be  interesting  when  presented  by  PLATO. 

36.  I  could  have  learned  more  if  I  hadn't  01  23456789 
felt  pushed. 

37.  I  was  given  answers  but  still  did  not  01  23456789 
understand  the  questions. 

38.  The  course  material  was  presented  too  0123456789 
slowly . 

39.  The  responses  to  my  answers  seemed  to  0123456769 
take  into  account  the  difficulty  of  the 

question . 

40.  While  on  PLATO,  I  encountered  mechanical  0123456789 
tnal  functions. 

41.  Computer-assisted  instruction  did  not  0123456789 
make  it  possible  for  me  to  learn  quickly. 

42.  PLATO  could  be  much  better  if  the  0123456789 

lessons  were  improved. 

43.  The  lessons  on  PLATO  were  interesting  0123456789 
and  really  kept  me  involved. 

44.  What  I  learned  from  PLATO  made  the  01  23456789 

classroom  and  laboratory  instruction 

easier  to  understand. 

45.  The  PLATO  lessons  were  dull  and  0123456789 

difficult  to  follow. 


PLATO  QUESTIONNAIRE 


SECTION  III 


Very  Very 

Strongly  Strongly 
Disagree _ Agree 

As  a  student,  I  do  my  best  work  with 

1.  Movies  and  filmstrips.  0 

2.  PLATO  lessons.  0 

3.  Study  guides  and  workbooks.  0 

4.  Instructor  lessons.  0 

5.  Laboratory  instructions.  0 

6.  Programmed  texts.  0 

'working  with  the  PLATO  system: 


7.  Is 

fun  . 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

8 .  Is 

frustrating. 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

9 .  Is 

challenging. 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10.  Is 

annoying. 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

11.  Is 

confusing. 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

12.  Makes  me  proud  of  myself. 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

13.  Is 

boring  . 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

14.  Is 

relax ing . 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

15.  Is 

depressing . 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

16.  Is 

enjoyable. 

0 

1 

o 

C. 

3 

4 

5 

6 

7 

8 

9 

17.  Is 

de-personal izing. 

0 

1 

2 

3 

4 

5 

6 

7 

3 

9 

13.  Is 

exiting. 

0 

1 

2 

3 

4 

5 

6 

7 

3 

9 

12  3456789 
123456789 
123456739 
1  23456739 
1  23456739 
1  23456739 
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t 
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10* 

Of  The 
Tine 


100* 
Of  The 
Time 


19.  How  often  has  PLATO  worked  when  you 
have  attempted  to  use  it? 

20.  During  how  many  sessions  have  the 

echanical  interruptions  made  you  want 
to  stop  using  PLATO? 


0123456789 


0123456789 


21.  During  your  school  day,  if  you  had  your  0  1  23456789 
choice,  how  much  of  your  time  would  be 
spent  working  with  PLATO? 
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I.  INTRODUCTION  HttCHHiC  PiOB  BUML-ii 


Background 


An  important  aspect  of  the  implementation  of  computer- 

based  education  (CBE)  is  the  attitude  of  instructors.  This 
report  examines  the  attitudes  of  instructors  at  Chanute  AFB, 
a  site  with  30  PLATO  IV  terminals.  These  attitudes  were 
shaped  by  the  instructors'  experience  while  observing 
students  as  they  proceeded  through  their  CBE  lessons  or 
while  working  at  PLATO  terminals  (i.e.,  "on-line")  them¬ 
selves.  The  significant  impact  of  the  instructor  attitudes 
on  the  pedagogical  effectiveness  of  the  lessons  is  noted 
(see  section  on  "Attitudes  of  Others"). 

These  attitudes  were  measured  in  several  ways.  MTC 
staff  interviewed  the  instructors  individually  and  in  a 
group.  This  investigator  administered  to  the  instructors  a 
questionnaire  regarding  the  specific  lessons  and  an  on-line 
survey  (Avner,  1976a)  on  attitudes  in  general.  In  addition, 
separate  reports  on  student  attitudes  in  the  vehicle  train¬ 
ing  course  have  been  prepared  by  Dallman,  DeLeo,  Main,  and 
Gillman  (1977),  and  by  Tatsuoka,  Misselt,  and  Maritz  (1978). 

Relevant  Factors 


Several  factors  contributed  to  the  formation  of  instruc¬ 
tor  attitudes: 


1.  the  ease  with  which  the  terminal  keyset  could  be 
used  (i.e.,  design/use  characteristics); 

2.  the  reliability  of  the  system  during  the  time  when 
the  students  were  scheduled  to  work; 

3.  the  quality  of  the  lesson  materials  being  used; 

4.  the  attitudes  of  fellow  instructors,  and  less 
directly,  other  individuals  who  influenced  the 
students'  feelings  toward  the  system; 

5.  the  quantitative  and  qualitative  benefits  the 
students  perceived  from  the  time  and  effort  invested 
in  study  of  the  lessons. 


Each  of  these  factors  is  considered  in  the  formulation  of  a 
comprehensive  estimate  of  instructor  attitudes. 


Expression  of  Attitudes 

Students  and  instructors  indicated  their  opinions  in 
various  ways: 


1.  orally  to  each  other  or  to  the  staff  during  or 
after  a  session  in  a  particular  lesson; 

2.  in  written  form  either  on  paper  (i.e.,  hard  copy) 
or  in  various  comment  files  maintained  on  the  PLATO 
system ; 

3.  in  a  non-verbal  fashion  by  their  action/inaction 
when  observers  from  CERL  were  present  (i.e.,  a 
student  spending  an  excessive  amount  of  time  on 
certain  frames)  the  instructors  could,  if  they  were 
so  inclined,  then  act  on  this  situation  by  calling 
these  frames  to  the  attention  of  the  authors. 

The  attitudes  of  the  teaching  and  support  staff,  coupled 
with  those  of  the  students,  helped  to  give  a  comprehensive 
perspective  of  the  total  learning  environment. 

II.  SPECIFIC  FACTORS  AFFECTING  ATTITUDES 


Inter  act  ion  with  the  Keyset  and  Terminal 

It  was  the  considered  opinion  of  the  instructor  staff 
that  students  had  little  trouble  working  with  the  keyset  at 
the  PLATO  terminal.  This  was  the  case  even  though  the 
students  taking  the  special  purpose  vehicle  and  general 
purpose  vehicle  courses  had  a  technical,  "hands-on" 
orientation  to  work.  As  one  former  instructor  put  it,  these 
students  would  rather  "have  a  wrench  in  their  hands," 
instead  of  a  keyset. 

The  instructor  staff  members  were  conscientious  and 
concerned  with  the  needs  of  students  interacting  with  PLATO. 
As  an  example,  when  some  of  the  students  complained  that  the 
semi-darkness  in  the  PLATO  classroom  (to  control  glare  on 
terminal  screens  from  overhead  lighting)  made  note-taking 
difficult,  a  small  high-intensity  lamp  was  installed  near 
each  terminal.  In  addition,  most  of  the  questions  within 
the  lessons  were  multiple  choice,  one  word,  or  short  answer 
in  format.  This  facilitated  the  interaction  with  the  term¬ 
inal  by  reducing  the  amount  of  typing  required.  These 
concerns  for  human  factors  tended  to  eliminate  any  diffi¬ 
culty  with  the  use  of  the  keyset. 

Rel  iabil ity  of  System 

In  the  first  few  years  of  PLATO  system  operation,  there 
was  considerable  concern  over  the  reliability  of  continuous 
service.  That  there  were  major  problems  at  that  time  cannot 
be  denied.  However,  the  quantity  and  duration  of 
interruptions  have  been  steadily  reduced  by  system  changes 
and  improvements,  both  in  software  and  hardware  technology. 
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In  addition,  careful  records  have  been  kept  since  1974 
listing  interruptions  of  service  during  prime  time  usage 
periods  as  well  as  reasons  for  the  interruptions  (Avner, 
1974).  Statistics  on  reliability  have  also  been  documented 
in  the  ARPA  annual  and  semi-annual  reports  (i.e.,  CERL, 

1975,  1976).  Nevertheless,  the  general  increase  in  trouble- 
free  operating  time  is  of  little  comfort  to  the  student  who 
experiences  a  crash  in  the  middle  of  a  lesson. 

When  there  is  an  interruption  of  system  operation,  the 
Chanute  instructors  wait  to  see  if  service  will  resume 
momentarily.  The  PLATO  system  staff  members  are  sometimes 
queried  by  telephone  regarding  the  expected  time  of 
resumption  of  service.  If  it  is  apparent  that  service  will 
not  be  restored  shortly,  the  instructors  take  the  students 
to  a  different  classroom  and  proceed  to  teach  (via  tradi¬ 
tional  lecture  and  discussion  methods)  the  materials  that 
would  have  been  covered  in  the  PLATO  materials  had  there 
been  no  interruption.  There  is  the  chance  that  the  instruc¬ 
tors  might  not  feel  prepared  to  take  over  the  class  on  a 
moment’s  notice,  and  thus  they  are  very  concerned  about  the 
probability  of  resumption  of  service.  In  addition,  there 
appears  to  be  little  enthusiasm  for  these  impromptu  lectures 
among  the  instructor  staff  even  though  some  may  have  been 
critical  of  PLATO  system  operation  in  general. 

System  interruptions  occur  for  a  variety  of  reasons  as 
a  result  of  software  and  hardware  problems.  During 
scheduled  weekday  periods,  there  is  usually  no  advance 
warning  before  an  interruption  or  "crash"  occurs.  However, 
authors  can  insert  special  commands  at  appropriate  places  in 
their  lessons  so  that,  once  the  system  is  working  again,  a 
student  can  continue  on  from  approximately  the  point  he  or 
she  had  reached  before  the  interruption  to  service.  This 
"restart"  feature  also  allows  students  to  leave  the  terminal 
either  for  a  break  or  at  the  end  of  the  day  and  automatically 
return  at  a  later  session  to  where  they  had  left  off. 

Observations  made  at  Chanute  show  that  these  precautions 
were  not  always  taken  by  the  authors  of  the  CBE  materials, 
and  even  when  implemented,  the  existence  or  use  of  these 
restart  points  was  apparently  not  understood  by  instructors. 
They  were  observed  informing  the  students  that  they  should 
leave  their  records  "signed-in"  while  the  students  left  the 
room  to  take  a  short  break.  The  reason  given  was  that  this 
procedure  would  keep  their  place  so  that  the  students  would 
not  have  to  repeat  a  large  portion  of  the  lesson  when  they 
resumed  work.  This  caution  on  the  part  of  the  instructors 
indicates  that  either  there  were  some  lessons  with 
inadequate  numbers  of  -restart-  points  or  that  the 
instructors  had  not  been  fully  trained  as  to  the  normal 
functioning  of  the  -restart-  points. 
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A  frustration  frequently  expressed  by  the  Chanute  staff 
members  is  that  they  received  insufficient  information  from 
PLATO  system  staff  when  they  inquired  about  the  reasons  for 
interruptions  of  service:  the  estimated  length  of  the  break 
in  service,  type  of  problem,  action  being  taken,  or  related 
matters.  The  instructors  found  no  relief  in  answers  that 
they  considered  vague  and  lacking  substantive  information. 
Because  they  reacted  rather  subjectively,  they  often  felt 
that  the  PLATO  computer's  operator  and  other  system  staff 
were  unaware  of  the  consequences  of  interrupted  service. 

The  instructors  gave  the  impression  that  they  felt  that 
Chanute  in  particular  was  being  victimized:  "Don't  they 
realize  we  have  a  classroom  full  of  students  up  here?"  was 
a  remark  made  by  several  of  the  staff  during  a  group 
interview. 

Leaving  aside  the  obvious  fact  that  central  system 
failures  affect  all  users,  not  merely  those  at  one  site, 
there  may  have  been  legitimate  grounds  here  for  expecting 
improved  feedback  about  the  anticipated  duration  of  the 
downtime.  Since  the  time  that  these  instructor  opinions 
were  gathered,  a  device  has  been  implemented  to  indicate 
this  information.  A  CERL  report  deals  with  the  problem  of 
reporting  reliability  (Avner,  1978). 

The  instructors  felt  that  system  failures  were  a 
serious  problem  for  students.  However,  there  was  little 
understanding  of  the  possible  reasons  for  these  inter¬ 
ruptions.  Even  the  usual  preventive  maintenance  time  (0600- 
07^0  every  day)  was  viewed  critically:  "Don't  they  know 
that  we  want  to  run  students  early  in  the  morning?"  might 
sum  up  their  collective  rejoinder.  This  same  feeling  about 
system  interruptions  in  general  was  expressed  in  an  on-line 
survey  (Avner,  1976a).  Fifteen  Chanute  instructors  were 
asked  to  respond  to  an  attitude  survey  using  this  same 
instrument:  of  the  ten  respondents,  six  instructors  felt 

that  system  interruptions  were  a  minor  problem  while  four 
considered  them  to  be  a  major  problem  (Table  1).  Therefore, 
it  is  clear  that  any  lost  time  in  service,  scheduled  or  not, 
resulted  in  a  very  negative  impression  on  the  Chanute  staff. 

The  actual  student  attitudes  toward  interruptions  of 
service  may  be  different  from  what  was  suggested  by  the 
instructors.  Preliminary  results  from  the  analysis  of 
student  questionnaires  indicate  that  the  influence  of  the 
crashes  may  not  have  been  as  substantial  as  previously 
expected.  For  more  details  on  student  attitude  results,  see 
Tatsuoka  et  al .  (1978). 


$ 

4 

1 

i 


51 


Table  1:  Survey  of  Author/Instructor  Attitudes 
torward  your  class. 


Suppose  in  the  future  you 

had  to 

teach 

the 

course  for 

which  you  had  the  most  PLATO  experience.  Would 

you  use 

PLATO  again? 

a.  never 

0 

b.  probably  not 

0 

c.  not  really  sure 

0 

d.  probably  would 

4 

e.  absolutely 

6 

Potential  problems  you  have  seen  with 

PLATO 

no 

prob 

minor 

prob 

major  prob 

1 . 

lessons  not  designed  for 
students 

6 

3 

1 

2. 

low  quality  of  lessons 

3 

6 

1 

3. 

lack  of  lessons  on  desired 
topics 

2 

5 

3 

4. 

system  crashes  in  the 
middle  of  classes  # 

0 

6 

4 

5. 

red-lighting  of  terminals 

2 

7 

1 

6. 

keeping  terminals  repaired 

0 

10 

0 

7. 

having  enough  terminals 
for  all  students 

0 

8 

2 

8. 

ECS  (can't  use  all  lessons 
needed  during  class) 

0 

5 

5 

9. 

basic  dehumanizing  charac¬ 
ter  of  computers 

8 

2 

0 

10. 

.  student  difficulties  in 

6 

4 

0 

use  of  equipment 

’Telephone  line  errors  interfering  with 

the 

transmission 

of 

student  key  presses  between 

the  terminals 

and 

the  central 

computer . 

Potential  benefits  of  using  PLATO 

no  evidence 

possible  ev 

.  clear  ev. 

1. 

gives  students  experi¬ 
ence  not  otherwise 

3 

6 

1 

available 

2. 

allows  better  teaching 
of  regular  material 

2 

4 

4 

3. 

permits  more  to  be 
covered 

1 

6 

3 

4. 

saves  time  of  students 

1 

2 

7 

5. 

saves  time  of  instructors 

0 

1 

9 

6. 

gives  instructors  better 
evidence  of  student  needs 

2 

6 

2 
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Quality  of  the  Lessons 

Notwithstanding  any  hardware  or  software  difficulties 
of  the  system,  the  lessons  themselves  have  made  a  very 
favorable  impact  on  the  students,  according  to  the  instruc¬ 
tors  at  Chanute.  During  a  group  interview,  the  instructors 
unanimously  agreed  that  the  lesson  quality  was  high  and 
covered  all  the  necessary  subject  matter  for  the  relevant 
areas.  A  similar  result  was  found  when  a  questionnaire 
on  specific  lessons  was  sent  to  each  indiviual  instructor; 
thirteen  instructors  were  asked  to  fill  out  the  question¬ 
naire.  Though  only  four  responded,  three  felt  that  the 
lessons  were  average  or  above  average  in  quality  (Table  2). 
As  additional  evidence,  an  unsolicited  conversation  with  a 
former  Chanute  student  has  confirmed  this  positive 
orientation  (personal  communication  to  a  CERL  staff 
member,  1976). 

During  MTC  observations  of  students,  some  lesson 
characteristics  were  noted  as  making  a  positive  impression 
on  the  students.  For  example,  the  complimentary  feedback 
after  a  correct  answer  to  a  question  (e.g.,  "Fantastic!") 
and  the  visual  impact  of  microfiche  images  were  seen  to 
generate  enthusiasm.  However,  students  sometimes  failed  to 
read  the  feedback  to  an  incorrect  response  and  they  did  not 
always  study  the  "HELP"  sequences  suggested  by  the  lesson's 
program.  The  old  bromide  about  leading  a  horse  to  water 
seems  appropriate  here. 

Table  2:  Ranking  of  Eight  Specific  Lessons^ 
Categories : 

1  =  unacceptable  as  it  stands 

2  =  acceptable  only  if  substantially  improved 

3  =  acceptable  as  is 

4  =  a  fairly  good  lesson 

5  =  one  of  the  best  lessons  I've  seen 

-  =  instructor  not  familiar  with  lesson 


emissions  (cha3) 

Inst .  A 

4 

Inst .  B 

3 

Inst.C 

4 

Inst .  D 

3 

Avg. 

3-5 

starters  ( cha4 1 ) 

5 

3 

3 

4 

3.7 

transmission(cha73) 

5 

— 

4 

4 

4.3 

diesels  (cha74) 

3 

2 

3 

4 

3.0 

hydraulics  (cha78) 

5 

2 

2 

5 

3.5 

drive  shaft  (cha82) 

3 

2 

4 

3 

3.0 

PTO  (cha86 ) 

3 

2 

3 

2 

2.5 

electricity  (cha97) 

5 

2 

2 

4 

3.2 

Averages 

4.  1 

2.3 

3.  1 

3.6 

3.3 
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(Table  2  cont.) 

Comments: 

1.  These  particular  lessons  were  selected  since 
detailed  reviews  of  them  had  been  prepared  by  the 
author  (Klecka,  1977a). 

2.  The  small  number  of  respondents  to  the  question¬ 
naire  makes  any  conclusions  drawn  from  it  very 
tentative . 

3.  The  data  available  indicate  that  most  instructors 
felt  the  lessons  were  at  least  satisfactory  in 
quality. 

4.  A  group  interview  with  Chanute  authors  and 
instructors  revealed  little  dissatisfaction  with  the 
lessons  except  for  a  lack  of  enthusiasm  for  the 
forced  review  technique. 

5.  The  group  interview  also  included  instructors  who 
did  not  respond  to  the  questionnaire  above.  The 
conclusions  drawn  from  that  interview  support  the 
data  above. 


^hese  lessons  were  critiqued  in  Klecka  (1977a). 


Students  were  frequently  observed  by  Chanute  and  MTC 
staff  to  be  taking  extensive  notes  while  going  through  the 
PLATO  lessons.  When  queried,  the  instructors  readily  agreed 
that  this  habit  was  not  only  tolerated  but  openly  encour¬ 
aged;  however,  the  checking  of  notebooks  by  the  instructors 
was  considered  optional,  according  to  a  Chanute  staff 
member.  Also,  in  some  cases  an  instructor  would  even  help  a 
student  take  notes  properly  so  that  he  would  not  continue  to 
copy  everything  on  the  screen.  When  questioned  about  this, 
several  instructors  suggested  that  the  students  felt  more 
comfortable  when  they  had  something  in  a  written  form  that 
they  could  take  with  them  for  later  review. 

This  feeling  of  satisfaction  when  their  well-annotated 
notebooks  were  at  hand  may  have  been  a  contributing  factor 
to  a  positive  attitude  toward  the  PLATO  system.  Also,  the 
lack  of  a  textbook  written  specifically  for  the  topics 
covered  in  the  CBE  lessons  may  have  been  significant  in 
encouraging  extensive  note-taking.  The  students  may  have 
felt  that  the  material  presented  on  the  screen  was  worth 
transcribing  and  retaining  for  future  reference.  In 
addition,  the  lessons  on  the  system  were  only  available  to 
students  during  their  CBE  class  per iods — they  had  no 
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opportunity  to  come  in  at  other  times  to  review  material  for 
added  comprehension.  All  these  factors  played  a  part  in  the 
extensive  note-taking,  and  there  were  few  objections  to  this 
practice  from  the  staff. 

Attitudes  of  Others 


In  past  studies  of  PLATO  usage,  the  attitude  of  the 
instructor  toward  the  computer  system  has  been  shown  to  have 
an  effect  on  student  attitudes  (Avner,  1976b).  Consequently, 
this  relationship  was  investigated  at  Chanute  in  the  present 
study.  When  instructors  were  interviewed,  they  stated  that 
they  were  all  very  interested  in  student  opinions  of  the 
PLATO  system  and  solicited  such  opinions  regularly.  They 
tried  to  keep  their  own  feelings  to  themselves  in  order  to 
avoid  coloring  student  attitudes,  but  in  general  they  had  a 
rather  positive  CBE  orientation. 

A  negative  attitude  in  particular  might  have  a  deleter¬ 
ious  effect  on  the  impact  of  the  materials  on  students.  If 
the  instructors  are  not  convinced  of  the  value  of  CBE,  it  is 
possiblr  that  this  feeling  will  be  transmitted  to  the 
students.  Since  it  is  unlikely  that  anyone  can  be  truly 
impartial  and  unbiased,  the  neutral  to  positive  attitude  of 
the  Chanute  instructors  is  preferred.  It  is  a  definite 
improvement  over  the  initial  lack  of  enthusiasm  that  was 
present  among  the  staff  members  (Klecka,  1977b;  Dallman,  et 
al.,  1977). 

In  spite  of  the  fact  that  most  of  the  classroom 
instruction  for  the  vehicle  maintenance  course  is  presented 
via  PLATO  lessons,  staff  members  must  still  be  present  with 
the  students  working  in  the  classroom.  The  instructors  or 
instructors'  assistants  (Dallman  et  al ,  ,  1977)  are  needed  to 
answer  any  questions  that  come  up  on  specific  points  in  the 
lessons  as  well  as  to  provide  alternative  explanations  for 
students  having  difficulty  with  the  standard  presentation. 
Even  though  they  felt  that  PLATO  instruction  provided  a 
standardized  format  (not  subject  to  the  vagaries  of 
instructors'  methods  on  different  days  of  teaching),  they 
agreed  that  there  is  still  room  for  each  student  to  get 
individualized  help  from  the  instructor  if  needed  and/or 
wanted . 

Contrast  the  above  picture  of  instructor  attitudes  with 
that  given  by  a  former  director  of  the  Instructional  Systems 
Development  group.  He  argued  that  the  group-paced  mode  of 
operation  is  the  worst  possible  for  instructors  since  the 
slowest  student  sets  the  pace.  The  instructor  cannot  sit 
down  with  the  slower  students  for  review  and  teach  them 
carefully — that  would  generally  mean  the  group  would  move  at 
a  still  slower  rate.  He  continued  by  saying  that  all  the 


instructor  can  do  is  prod  the  student,  or  sit  at  the 
terminal  and  give  him  the  answers  so  the  group  can  move  on. 
Further,  the  instructor  is  no  longer  the  center  of  attention 
in  the  classroom,  no  matter  what  the  subject  of  his  lectures 
is  (related  to  the  instructional  objectives  or  otherwise). 
The  loss  of  the  position  of  authority  might  reasonably  tend 
to  make  the  instructors  less  enthusiastic  about  the  PLATO 
system . 

Remarkably,  the  attitudes  which  the  MTC/PEER  Group 
found  when  interviewing  instructors  did  not  reflect  the 
pessimism  suggested  by  the  discussion  above  nor  the 
instructor  dissatisfaction  found  by  AFHRL  evaluators 
(Dallman  et  al . ,  1977).  This  discrepancy  may  be  due  to  the 
time  variation  at  which  the  attitude  measurements  were  made. 
The  AFHRL  evaluators  gathered  their  data  in  February  and 
July  of  1975  and  the  comment  from  the  ISD  chief  was  made  in 
the  summer  of  1975.  In  contrast,  the  data  for  this  report 
were  gathered  in  October,  1976.  Some  of  the  problems  with 
the  lessons  may  have  been  worked  out  along  with  increasing 
familiarity  with  the  system  by  the  instructors — these 
developments  may  have  helped  to  improve  their  attitudes  in 
time . 

There  was  general  agreement  among  the  author  and 
instructor  staff  that  providing  more  Extended  Core  Storage 
(ECS  or  computer  memory  space)  for  accessing  lessons  during 
times  of  peak  usage  would  help  alleviate  some  difficulties. 
Certain  problems  with  inadequate  storage  space  mentioned  by 
the  instructors  have  also  been  observed  by  MTC  staff: 
students  frequently  were  unable  to  proceed  from  one  lesson 
to  another  when  ECS  usage  exceeded  the  base  allotment 
guaranteed  to  the  Chanute  site  for  their  use.  It  appears 
likely  that  these  problems  could  be  remedied  by  adding  more 
ECS  or  similar  memory  space  to  the  computer  or  by  scheduling 
PLATO  courses  at  other  than  peak  usage  time,  a  less  likely 
alternative . 

Another  possibility  is  dividing  the  larger  lessons  into 
smaller  segments  to  fit  more  easily  into  the  limited  ECS 
allocation.  Doing  this  would  also  alleviate  the  memory 
shortage.  As  a  consequence,  it  seems  probable  that  an 
increase  in  available  space  for  accessing  lessons  would 
improve  instructor  and  possibly  student  attitudes.  Of  the 
ten  instructors  responding  to  the  MTC  survey,  five  felt  that 
insufficient  ECS  was  a  minor  problem  while  the  remainder 
considered  it  a  major  problem  (Table  1). 

Interviews  have  suggested  that  instructor  attitudes  may 
have  been  influenced  by  an  upper-level  administrator  who  was 
reported  to  be  reluctant  to  accept  the  computer  as  a 
suitable  vehicle  for  delivering  effective  instruction.  This 
reluctance  was  perceived  by  the  instructors  and  the 
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authoring  staff  as  a  tendency  by  the  administrator  in 
question  to  simply  mark  time  in  hopes  that  the  whole  CBE 
program  at  Chanute  would  just  fade  away.  The  chief 
administrator  was  not  interviewed  so  that  an  independent 
assessment  of  his  attitudes  cannot  be  given.  However,  his 
lack  of  apparent  support  was  widely  perceived  among  those 
closest  to  the  project. 

No  evidence  of  any  "ripple  effect"  was  detected  by  this 
investigator  among  the  instructors,  but  several  lesson 
authors  indicated  this  administrative  feeling  made  them 
apprehensive  about  their  status. 

perceived  Benefits  of  System 

Some  indications  of  lesson  quality  can  be  gained  from 
the  previously-mentioned  questionnaire  as  well  as  interviews 
with  the  instructors  themselves.  Although  not  specifically 
stated  by  the  instructors,  it  might  be  correctly  assumed 
that  not  only  what  is  presented  (i.e.,  technical  subjects), 
but  also  the  interactive  format  in  which  this  material  is 
displayed  plays  an  important  role  in  the  positive  benefits 
that  can  be  gained.  Also,  it  was  suggested  by  the 
instructor  staff  in  an  interview,  but  not  actually  observed 
by  them,  that  a  student  might  be  more  inclined  to  ask  a 
question  at  the  terminal  (e.g.,  the  definition  of  an 
unfamiliar  term)  than  he  would  in  the  classroom  before  the 
critical  gaze  of  his  peers.  If  the  CBE  tutorial  format  does 
make  students  less  inhibited  about  the  subject  matter,  it  in 
fact  is  fulfilling  a  very  useful  role. 

It  cannot  be  denied  that  recreational  uses  of  the 
computer  contribute  to  students’  positive  feelings  toward 
the  PLATO  system.  In  fact,  the  authors  and/or  instructors 
had  to  enforce  a  policy  that  recreational  uses  of  the  system 
were  not  allowed  during  the  class  period  since  they 
interfered  with  progress  through  the  assigned  lessons--the 
students  would  rush  through  the  material  in  order  to  begin 
playing  games,  according  to  Chanute  staff.  This  inclination 
to  utilize  the  system  resources  for  recreational  uses  may 
have  been  a  factor  at  an  early  stage  for  a  very  casual 
attitude  on  the  part  of  the  instructors  toward  CBE:  it  was 
satisfactory  for  playing  games  but  perhaps  not  for  more 
serious  instructional  purposes. 

The  extent  to  which  the  students  were  cognizant  of  the 
individualizing  character  of  PLATO  lessons  can  only  be 
measured  approximately.  However,  some  of  the  previous 
comments  on  their  reaction  to  feedback  indicate  that  they 
can  appreciate  the  individualizing  aspects  of  a  CBE  lesson 
when  these  are  encountered.  Nevertheless,  there  is  a 
question  as  to  the  difficulty  of  the  reading  level  of 
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particular  lessons.  Some  instructors  felt  that  the  students 
could  read  at  an  acceptable  level  to  understand  all  the 
lessons,  while  others  were  less  certain. 

Conveniently,  as  part  of  another  study,  an  analysis  of 
the  reading  levels  in  selected  lessons  was  undertaken 
("Readability  study"  by  R.  A.  Avner  in  Klecka,  1977a).  In 
that  study  it  was  determined  that  the  reading  level  in  some 
was  approximately  three  grade  levels  higher  than  the 
reading  level  of  a  typical  Chanute  student.  Thus  the  obser¬ 
vations  of  the  instructors  to  the  effect  that  the  lessons 
were  of  an  appropriate  level  of  readability  seem  to  be 
confirmed.  Unpublished  studies  by  the  Military  Training 
Centers  and  PLATO  Educational  Evaluation  and  Research  group 
staffs  indicated  that  the  reading  level  variations  from 
lesson  to  lesson  apparently  have  a  relatively  small  effect 
on  a  broad  range  of  performance  variables. 


III.  MONITORING  and  RECORDING  of  STUDENT/INSTRUCTOR  COMMENTS 

The  instructor  opinions  discussed  above  were  shaped 
through  lengthy  interaction  with  the  students  in  their 
courses.  These  students  expressed  their  opinions  either 
orally  as  they  went  through  their  assignments  or  in  written 
comments  solicited  at  the  end  of  the  course.  The  communi¬ 
cation  of  these  remarks  will  be  discussed  in  more  detail 
since  they  furnished  the  foundation  for  much  of  the  material 
in  this  report. 

Oral  Remarks 


As  they  went  through  their  assignments,  the  students 
were  quite  expressive  to  their  peers  and  to  the  instructors 
on  various  aspects  of  the  lessons  and  system.  For  example, 
a  system  crash  would  occasion  negative  comments  which  were 
vocalized  loudly  enough  to  be  heard  by  the  instructors 
present.  Also,  particular  frames  of  a  lesson  might  elicit 
questions  from  the  student  or  even  exclamations  on  some 
aspect  of  that  particular  frame  or  interaction.  Thus,  the 
instructor  present  in  the  classroom  was  in  an  excellent 
position  to  collect  and  interpret  these  informative 
comments . 

Written  Comments 


Two  avenues  were  available  for  students  to  record  their 
comments  in  written  form: 

1.  a  notes  file  available  while  the  student. was  on¬ 
line,  or 
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2.  AF  form  736  administered  at  the  end  of  whole  course 
of  instruction. 

In  addition,  an  instructor  notes  ile  ( "chanotes" )  was 
available  for  staff  use  to  insert  specific  comments  on 
particular  lessons.  Each  method  furnished  different  types 
of  information  regarding  the  course,  particular  lessons,  or 
the  instructor  staff. 

Studenb  notes  file.  Use  of  the  student  notes  file 
( "chastuden t" )  after  January  1,  1977,  served  to  open  up 
another  means  for  obtaining  feedback  from  students.  It  was 
accessible  by  students  at  any  point  in  the  lessons  to  make 
comments  of  either  a  general  or  specific  nature.  Appro¬ 
priate  encouragement  from  the  instructors  and  an  observed 
willingness  on  the  part  of  the  students  to  cooperate 
indicated  that  this  means  of  communication  about  lesson 
problems  would  be  useful  in  making  the  needed  changes. 

Points  of  high  student  frustration  can  be  readily 
identified  through  the  student  notes  mechanism  because  notes 
initiated  by  students  are  automatically  labeled  with  loca¬ 
tion  information.  At  the  time  of  this  writing,  the  use  of 
the  lesson  notes  feature  at  Chanute  was  still  in  the  early 
stages  of  implementation.  Some  of  the  comments  were  gener¬ 
ally  favorable,  but  they,  like  some  derisive  ones,  were  not 
specific  enough  to  be  useful  i.e.,  they  didn't  identify 
strategies  that  were  well-liked  and  hence  should  be  repli¬ 
cated.  Some  more  recent  notes,  however,  did  pertain  to 
specific  aspects  of  the  lessons  and  may  be  worth  consi¬ 
dering.  Examples  of  the  latter  are  given  below,  with  the 
names  changed  to  protect  anonymity. 


Figure  1:  Examples  of  Useful  Student  Notes 

chastudent  2/10/77  11:5*1  am  harris/  spvehl 

lesson:  mrouter  site:  chanute  10-11 

Concerning  the  section  on  transmissions  there  should  be 
either  plato  pictures  or  slides  on  the  location  and  opera¬ 
tion  of  turbines  and  stator.  Diagrams  are  needed  for  a 
proper  understanding  of  the  operation  of  transmissions. 

chastudent  2/15/77  11:54  am  blake/  spvehl 

lesson:  mrouter  site:  chanute  10-11 

In  the  basic  electricity  course,  there  aren't  enough  exam¬ 
ples  on  parallel  and  series-parallel  circuits.  So  I  feel 
you  should  correct  your  lesson  plan. 


Course  critique  form.  The  regulation  AF  course 
critique  form  (AF  form  736)  furnished  only  general  comments 
on  the  particular  course,  and  it  was  not  very  useful  for 
locating  specific  problems  within  the  lessons  or  in  teaching 
strategies.  Also,  the  time  span  between  completing  the 
lessons  and  filling  out  the  form  made  the  recall  of  specific 
comments  difficult.  The  course  critiques  are  all  given  at 
the  end  of  the  entire  course,  according  to  a  staff  member. 
Although  any  comments  are  usually  better  than  none  at  all, 
the  data  presented  in  these  forms  were  difficult  to  evaluate 
due  to  their  general  nature. 

When  instructors  were  queried  regarding  the  usefulness 
of  these  critiques,  there  was  limited  interest  in  them  as  a 
tool  for  corrective  action.  It  was  felt  by  the  evaluator 
that  they  were  administered  because  AF  regulations  required 
them;  but  no  enthusiasm  was  evident  for  analyzing  their 
contents.  This  feeling  is  probably  justifiable  considering 
the  format  as  well  as  the  timing:  the  form  is  administered 
when  students  have  already  completed  the  course  and  hence 
have  little  additional  interest  in  it. 

Instructor  notes  file.  A  fairly  late  development  in 
the  operation  of  the  PLATO-based  Training  System  at 
Chanute  was  the  use  of  a  group  notes  file  ("chanotes")  for 
instructors  to  put  in  their  comments  on  lessons.  These 
notes  were  read  by  the  Chanute  CBE  authoring  staff  and  the 
recommendations  for  changes  in  the  lessons  were  taken  under 
advisement.  This  file  offered  another  avenue  of  recording 
and  acting  on  relevant  remarks. 

Instructors  were  also  able  to  make  comments  on 
individual  test  items  and  specific  student  responses  to 
these  items  by  the  use  of  various  data  collection  and 
management  files.  For  detailed  information  on  this  subject, 
see  "Computer-Aided  Management  at  Chanute,"  Chapter  2  of 
Klecka  (1977b). 


IV.  CONCLUSIONS 

As  in  any  situation,  the  attitude  of  the  users  and  thus 
the  benefits  derived  from  the  system  can  be  improved  by 
implementation  of  certain  changes.  It  is  hoped  that  this 
process  will  be  aided  by  the  following  conclusions: 

1.  interruptions  of  system  service  had  a  negative 
effect  on  the  attitudes  of  the  instructors; 

2.  virtually  all  instructors  surveyed  felt  that  the 
Chanute  lessons  were  of  average  or  above  average 
pedagogical  quality; 
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3.  the  attitudes  of  instructors  did  have  a  generally 
favorable  effect  on  the  students,  according  to 
Chanute  staff; 

4.  in  general,  according  to  the  instructors,  the 
students  had  a  positive  attitude  toward  the  PLATO 
system,  although  there  was  still  some  residue  of 
viewing  it  as  experimental  which  may  have  induced  a 
slightly  more  casual  attitude; 

5.  up  to  the  time  of  the  writing  of  this  report, 
student  comments  delivered  orally  tended  to  be  more 
specific  and  therefore  more  useful  for  lesson 
revision  and  pedagogical  modifications  than  those 
written  down  in  course  critique  forms  (AF  form  736); 

6.  the  student  notes  and  lesson  file  capability 
offered  the  opportunity  for  improvement  of  lessons 
based  on  a  broader  range  of  specific  comments. 
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